Lens apparatus, image pickup apparatus, processing apparatus, processing method, and computer-readable storage medium

ABSTRACT

A lens apparatus includes an optical member, a driving device configured to perform driving of the optical member, a detector configured to detect a state related to the driving, and a processor configured to generate a control signal for the driving device based on first information about the detected state, wherein the processor includes a machine learning model configured to generate an output related to the control signal based on the first information and second information about the lens apparatus, and is configured to output the first information and the second information to a generator configured to perform generation of the machine learning model.

BACKGROUND OF THE DISCLOSURE Field of the Disclosure

The aspect of the embodiments relates to a lens apparatus, an imagepickup apparatus, a processing apparatus, a processing method, and acomputer-readable storage medium.

Description of the Related Art

Some recent digital cameras can capture not only still images but alsomoving images. For quick still-image capturing capability, high-speedautomatic focusing, zooming, and aperture operations are required. Bycontrast, in capturing a moving image, high operation noise from adriving system for the high-speed operations can impair the quality ofthe sound recorded along with the image. In view of this, JapanesePatent Application Laid-Open No. 2007-006305 discusses an image pickupapparatus that switches an operation mode of its actuators to a silentmode during moving image capturing.

A wide variety of types of performance are required of the actuators fordriving the optical members of an image pickup apparatus. Examplesinclude performance about driving speed related to controlfollowability, positioning accuracy related to accurate imagingcondition settings, power consumption related to continuous image-pickupduration, and quietness related to the quality of sound during movingimage capturing. These types of performance are mutually dependent. Forexample, the image pickup apparatus discussed in Japanese PatentApplication Laid-Open No. 2007-006305 improves quietness by limitingdriving speed and acceleration.

Desirable quietness can vary depending on the imaging situation.Desirable driving speed and acceleration can also vary depending on theimaging situation. The same applies to other types of performance suchas the positioning accuracy and the power consumption. Moreover,priorities of the respective types of performance can vary depending onthe imaging situation and the operator. Thus, the actuators aredesirably operated with driving performance suitable for various imagingsituations and operators.

SUMMARY OF THE DISCLOSURE

According to an aspect of the embodiments, a lens apparatus includes anoptical member, a driving device configured to perform driving of theoptical member, a detector configured to detect a state related to thedriving, and a processor configured to generate a control signal for thedriving device based on first information about the detected state,wherein the processor includes a machine learning model configured togenerate an output related to the control signal based on the firstinformation and second information about the lens apparatus, and isconfigured to output the first information and the second information toa generator configured to perform generation of the machine learningmodel.

Further features of the disclosure will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating a configuration example of a lensapparatus according to a first exemplary embodiment.

FIGS. 2A and 2B are diagrams illustrating positioning accuracy servingas driving performance.

FIGS. 3A and 3B are diagrams illustrating driving speed serving asdriving performance.

FIGS. 4A and 4B are diagrams illustrating a relationship of thepositioning accuracy with the driving speed, power consumption, andquietness.

FIGS. 5A and 5B are diagrams illustrating a relationship of the drivingspeed with the positioning accuracy, the power consumption, and thequietness.

FIG. 6 is a diagram illustrating inputs and an output of a neuralnetwork.

FIG. 7 is a flowchart illustrating a processing procedure of machinelearning.

FIGS. 8A1, 8A2, 8B1, 8B2, 8C1, 8C2, 8D1 and 8D2 are diagramsillustrating reward information.

FIG. 9 is a diagram illustrating a data structure of reward information.

FIGS. 10A to 10C are diagrams illustrating data structures ofinformation about options for a second reward section.

FIG. 11 is a diagram illustrating a configuration example of a lensapparatus according to a second exemplary embodiment.

FIG. 12 is a diagram illustrating inputs and an output of a neuralnetwork.

FIGS. 13A1, 13A2, 13B1 and 13B2 are diagrams illustrating rewardinformation.

FIG. 14 is a diagram illustrating a data structure of rewardinformation.

FIGS. 15A and 15B are diagrams illustrating data structures ofinformation about options for a second reward section.

FIG. 16 is a diagram illustrating a configuration example of a lensapparatus according to a third exemplary embodiment.

DESCRIPTION OF THE EMBODIMENTS

Exemplary embodiments of the disclosure will be described below withreference to the attached drawings. Throughout the drawings fordescribing the exemplary embodiments, similar members are denoted by thesame reference numerals in principle (unless otherwise specified). Aredundant description thereof will be omitted.

First Exemplary Embodiment

«Configuration Example Where Camera Main Body (Processing Apparatus)Includes Training Unit (Generator)»

FIG. 1 is a diagram illustrating a configuration example of a lensapparatus according to a first exemplary embodiment, and by extension, adiagram illustrating a configuration example of a system (image pickupapparatus) including a configuration example of a camera main body (alsoreferred to as a camera apparatus main body, an image pickup apparatusmain body, or a processing apparatus) as well. The system includes acamera main body 200 and a lens apparatus 100 (also referred to as aninterchangeable lens) mounted on the camera main body 200. The cameramain body 200 and the lens apparatus 100 are mechanically andelectrically connected via a mount 300 serving as a coupling mechanism.The mount 300 may be composed of a mount unit belonging to the cameramain body 200 and a mount unit belonging to the lens apparatus 100, ormay be configured to include both of the mounting units. The camera mainbody 200 can supply power to the lens apparatus 100 via a power supplyterminal included in the mount 300. The camera main body 200 and thelens apparatus 100 can communicate with each other via a communicationterminal included in the mount 300. In the present exemplary embodiment,the lens apparatus 100 and the camera main body 200 are configured to beconnected via the mount 300. However, the lens apparatus 100 and thecamera main body 200 may be integrally configured without a mount.

The lens apparatus 100 can include a focus lens unit 101 for changing anobject distance, a zoom lens unit 102 for changing a focal length, anaperture stop 103 for adjusting an amount of light, and an imagestabilization lens unit 104 intended for image stabilization. The focuslens unit 101 and the zoom lens unit 102 are held by respective holdingframes. The holding frames are configured to be movable in the directionof an optical axis (the direction of the broken line in the diagram) viaguide shafts, for example. The focus lens unit 101 is driven along thedirection of the optical axis by a driving device 105. A detector 106detects the position of the focus lens unit 101. The zoom lens unit 102is driven along the direction of the optical axis by a driving device107. A detector 108 detects the position of the zoom lens unit 102. Theaperture stop 103 includes diaphragm blades. The diaphragm blades aredriven by a driving device 109 to adjust the amount of light. A detector110 detects an opening amount (also referred to as a degree of openingor f-number) of the aperture stop 103. The image stabilization lens unit104 is driven by a driving device 112 in directions including componentsorthogonal to the optical axis, whereby image shakes due to camerashakes are reduced. A detector 113 detects the position of the imagestabilization lens unit 104. The driving devices 105, 107, 109, and 112can be configured to include an ultrasonic motor, for example. Thedriving devices 105, 107, 109, and 112 are not limited to ultrasonicmotors, and may be configured to include other motors such as a voicecoil motor, a direct-current (DC) motor, and a stepping motor.

The detectors 106, 108, 110, and 113 can be configured to include apotentiometer or an encoder, for example. If a driving device includes amotor capable of driving by a given driving amount without feedback ofthe driving amount (control amount), such as a stepping motor, then adetector for detecting a specific position (a reference position or apoint of origin) may be provided. In such a case, the detector caninclude a photo-interrupter, for example. A detector 111 detects shakesof the lens apparatus 100. The detector 111 can include a gyroscope, forexample.

A processor 120 can be a microcomputer, and can include an artificialintelligence (AI) control unit 121, a determination unit 122, a storageunit 123, a log storage unit 124, a driving control unit 125, and acommunication unit 126. The AI control unit 121 is a control unit thatcontrols driving of the focus lens unit 101. The AI control unit 121here can operate based on a neural network (NN) algorithm. In morecommon terms, the AI control unit 121 generates a driving instructionfor the driving device 105 of the focus lens unit 101 by using a machinelearning model. The determination unit 122 is a determination unit thatdetermines information about the lens apparatus 100 (second information)for the AI control unit 121 to use. The storage unit 123 is a storageunit that stores information for identifying the type of input (featureamount) to the NN, and information about weights assigned to inputs torespective layers. The log storage unit 124 stores information about anoperation log of the lens apparatus 100 concerning the driving controlon the focus lens unit 101. The driving control unit 125 controlsdriving of the zoom lens unit 102, the aperture stop 103, and the imagestabilization lens unit 104. For example, the driving control unit 125can generate a driving instruction for the driving devices 107, 109, and112 by proportional-integral-derivative (PID) control based ondeviations between target positions or target speeds of objects to becontrolled and the actual positions or actual speeds of the objects tobe controlled. The communication unit 126 is a communication unit forcommunicating with the camera main body 200. The NN algorithm, theweights, the second information, and the operation log will be describedbelow.

The camera main body 200 (processing apparatus) can include an imagepickup element 201, an analog-to-digital (A/D) conversion unit 202, asignal processing circuit 203, a recording unit 204, a display unit 205,an operation device 206, a processor 210 (also referred to as a cameramicrocomputer), and a training unit 220. The image pickup element 201picks up an image formed by the lens apparatus 100. For example, theimage pickup element 201 can include a charge-coupled device (CCD) imagesensor or a complementary metal-oxide-semiconductor (CMOS) device. TheA/D conversion unit 202 converts an analog signal (image signal)captured and output by the image pickup element 201 into a digitalsignal. The signal processing circuit 203 converts the digital signaloutput from the A/D conversion unit 202 into image data. The recordingunit 204 records the image data output from the signal processingcircuit 203. The display unit 205 displays the image data output fromthe signal processing circuit 203. The operation device 206 is intendedfor an operator (user) to operate the image pickup apparatus.

The processor 210 is intended to control the camera main body 200, andcan include a control unit 211 and a communication unit 212. The controlunit 211 generates a driving instruction for the lens apparatus 100based on the image data from the signal processing circuit 203 and theoperator's input information from the operation device 206. The controlunit 211 also gives an instruction and transmits information to thetraining unit 220 (to be described below). The communication unit 212communicates with the lens apparatus 100. The communication unit 212transmits the driving instruction from the control unit 211 to the lensapparatus 100 as a control command. The communication unit 212 alsoreceives information from the lens apparatus 100.

The training unit 220 (generator) can include a processor (such as acentral processing unit (CPU) and a graphics processing unit (GPU)) anda storage device (such as a read-only memory (ROM), a random accessmemory (RAM), and a hard disk drive (HDD)). The training unit 220 caninclude a machine learning unit 221, a reward storage unit 223, a firstreward section storage unit 224, a second reward section storage unit225, and a log storage unit 222. The training unit 220 also stores aprogram for controlling operation of the units 221 to 225. Rewardinformation stored in the reward storage unit 223, information about afirst reward section stored in the first reward section storage unit224, information about a second reward section stored in the secondreward section storage unit 225, and information for obtaining theinformation about the second reward section from information input bythe operator will be described below.

<Recording and Display of Image Data>

The recording and display of image data by the image pickup apparatusillustrated in FIG. 1 will now be described.

Light entering the lens apparatus 100 forms an image on the image pickupelement 201 via the focus lens unit 101, the zoom lens unit 102, theaperture stop 103, and the image stabilization lens unit 104. The imagepickup element 201 converts the image into an electrical analog signal.The A/D conversion unit 202 converts the analog signal into a digitalsignal. The signal processing circuit 203 converts the digital signalinto image data. The image data output from the signal processingcircuit 203 is recorded in the recording unit 204. The image data isalso displayed on the display unit 205.

<Focus Control>

Next, focus control of the lens apparatus 100 by the camera main body200 will be described. The control unit 211 performs automatic focus(AF) control based on the image data output from the signal processingcircuit 203. For example, the control unit 211 performs AF control todrive the focus lens unit 101 so that the contrast of the image data ismaximized. The control unit 211 outputs a driving amount of the focuslens unit 101 to the communication unit 212 as a driving instruction.The communication unit 212 receives the driving instruction from thecontrol unit 211, converts the driving instruction into a controlcommand, and transmits the control command to the lens apparatus 100 viacommunication contact members of the mount 300. The communication unit126 receives the control command from the communication unit 212,converts the control command into a driving instruction, and outputs thedriving instruction to the AI control unit 121 via the driving controlunit 125. As the driving instruction is input, the AI control unit 121generates a driving signal based on a machine learning model (trainedweights) stored in the storage unit 123, and outputs the driving signalto the driving device 105. Details of generation of the driving signalby the AI control unit 121 will be described below. In such a manner,the focus lens unit 101 is driven based on the driving instruction fromthe control unit 211 of the camera main body 200. Thus, the control unit211 can perform the AF control to drive the focus lens unit 101 so thatthe contrast of the image data is maximized.

<Aperture Stop Control>

Next, aperture stop control of the lens apparatus 100 by the camera mainbody 200 will be described. The control unit 211 performs aperture stopcontrol (exposure control) based on the image data output from thesignal processing circuit 203. Specifically, the control unit 211determines a target f-number so that the image data has a constantluminance value. The control unit 211 outputs the determined f-number tothe communication unit 212 as a driving instruction. The communicationunit 212 receives the driving instruction from the control unit 211,converts the driving instruction into a control command, and transmitsthe control command to the lens apparatus 100 via the communicationcontact members of the mount 300. The communication unit 126 receivesthe control command from the communication unit 212, converts thecontrol command into a driving instruction, and outputs the drivinginstruction to the driving control unit 125. As the driving instructionis input, the driving control unit 125 determines a driving signal basedon the driving instruction and the f-number of the aperture stop 103detected by the detector 110, and outputs the driving signal to thedriving device 109. In such a manner, the aperture stop 103 is driven tomake the luminance value of the image data constant based on the drivinginstruction from the control unit 211 of the camera main body 200. Thus,the control unit 211 can perform exposure control to drive the aperturestop 103 so that the exposure amount of the image pickup element 201 isappropriate.

<Description of Zoom Control>

Next, zoom control of the lens apparatus 100 by the camera main body 200will be described. The operator performs a zoom operation on the lensapparatus 100 via the operation device 206. The control unit 211 outputsthe driving amount of the zoom lens unit 102 to the communication unit212 as a driving instruction based on an amount of the zoom operationoutput from the operation device 206. The communication unit 212receives the driving instruction, converts the driving instruction intoa control command, and transmits the control command to the lensapparatus 100 via the communication contact members of the mount 300.The communication unit 126 receives the control command from thecommunication unit 212, converts the control command into a drivinginstruction, and outputs the driving instruction to the driving controlunit 125. As the driving instruction is input, the driving control unit125 generates a driving signal based on the driving instruction and theposition of the zoom lens unit 102 detected by the detector 108, andoutputs the driving signal to the driving device 107. In such a manner,the zoom lens unit 102 is driven based on the driving instruction fromthe control unit 211 of the camera main body 200. Thus, the control unit211 can perform zoom control to drive the zoom lens unit 102 based onthe amount of the zoom operation output from the operation device 206.

<Image Stabilization Control>

Next, image stabilization control of the lens apparatus 100 will bedescribed. The driving control unit 125 determines a target position ofthe image stabilization lens unit 104 to reduce image shakes due tovibrations of the lens apparatus 100 based on a signal indicating thevibrations of the lens apparatus 100, output from the detector 111. Thedriving control unit 125 generates a driving signal based on the targetposition and the position of the image stabilization lens unit 104detected by the detector 113, and outputs the driving signal to thedriving device 112. In such a manner, the image stabilization lens unit104 is driven based on the driving signal from the driving control unit125. Thus, the driving control unit 125 can perform image stabilizationcontrol to reduce image shakes due to vibrations of the lens apparatus100.

<Driving Performance Related to Focus Control>

Four types of driving performance related to focus control, namely,positioning accuracy, driving speed, power consumption, and quietnesswill be described. These types of driving performance are adapted tovarious situations where focus control is performed.

(1) Positioning Accuracy

The positioning accuracy will be described with reference to FIGS. 2Aand 2B. FIGS. 2A and 2B are diagrams illustrating the positioningaccuracy serving as driving performance. FIGS. 2A and 2B illustratecases where the depth of focus is small and where the depth of focus islarge, respectively. In FIGS. 2A and 2B, the f-numbers are different. Atarget position G of the focus lens unit 101 represents the position ofthe focus lens unit 101 where a point-like object S on the optical axisis in focus on the image pickup element 201. A position C represents theactual position of the focus lens unit 101 after the focus lens unit 101is driven to the target position G. The position C is on an object Sside of the target position G by a control error (control deviation) E.An image forming position (focal point position) Bp represents theposition where the image of the object S is formed when the focus lensunit 101 is located at the position C. The image pickup element 201 hasa permissible circle of confusion (diameter) δ.

The f-number (Fa) in FIG. 2A is smaller (brighter) than the f-number(Fb) in FIG. 2B. Thus, the depth of focus (2Faδ) in FIG. 2A is smallerthan the depth of focus (2Fbδ) in FIG. 2B. Rays Ca and rays Ga in FIG.2A represent the outermost rays from the object S when the focus lensunit 101 is located at the position C and the target position G,respectively. Rays Cb and rays Gb in FIG. 2B represent the outermostrays from the object S when the focus lens unit 101 is located at theposition C and the target position G, respectively. In FIG. 2A, thepoint image of the object S on the image pickup element 201 when thefocus lens unit 101 is located at the position C has a diameter Ia. InFIG. 2B, the point image of the object S on the image pickup element 201when the focus lens unit 101 is located at the position C has a diameterIb.

In FIG. 2A, the focal point position Bp falls outside the depth of focus(2Faδ). The diameter Ia of the point image is greater than thepermissible circle of confusion δ, and the point image goes beyond thecenter pixel of the image pickup element 201 and spreads to adjoiningpixels. Thus, in FIG. 2A, with the focus lens unit 101 at the positionC, the object S is in an out-of-focus state. By contrast, in FIG. 2B,the focal point position Bp falls within the depth of focus (2Fbδ). Thediameter Ib of the point image is smaller than the permissible circle ofconfusion δ, and the point image lies within the center pixel of theimage pickup element 201. Thus, in FIG. 2B, with the focus lens unit 101at the point C, the object S is in an in-focus state. Thus, given thesame positioning accuracy of the focus lens unit 101, the in-focus stateis either able or not able to be achieved depending on the imagingcondition. In other words, a desirable positioning accuracy changes withthe imaging condition.

(2) Driving Speed

The driving speed refers to an amount of movement per unit time. A focalpoint moving speed refers to an amount of movement of the focal pointper unit time. An amount of movement of the focus lens unit 101 isproportional to the amount of movement of the focal point. Aproportionality constant in this proportional relationship will bereferred to as a focus sensitivity. In other words, the focussensitivity is the amount of movement of the focal point of the lensapparatus 100 per unit amount of movement of the focus lens unit 101.The focus sensitivity varies depending on the state of an optical systemconstituting the lens apparatus 100. An amount of movement of the focalpoint ΔBp can be expressed by the following Eq. (1):ΔBp=Se×ΔP,  (1)where Se is the focus sensitivity, and ΔP is the amount of movement ofthe focus lens unit 101.

The driving speed required for focus control will now be described withreference to FIGS. 3A and 3B. FIGS. 3A and 3B are diagrams illustratingthe driving speed serving as driving performance. FIGS. 3A and 3Billustrate cases where the focus sensitivity Se is high and where thefocus sensitivity Se is low, respectively. In FIGS. 3A and 3B, theobject distances are different. In FIG. 3A, the position of the focuslens unit 101 is moved from position Pa1 to position Pa2 in moving theposition of the focal point from position Bp1 to position Bp2. Therelationship between the amount of movement ΔPa (ΔP) of the focus lensunit 101 and the amount of movement of the focal point ΔBp here is givenby Eq. (1). In FIG. 3B, the position of the focus lens unit 101 is movedfrom position Pb1 to position Pb2 in moving the position of the focalpoint from position Bp1 to position Bp2. The relationship between theamount of movement ΔPb (ΔP) of the focus lens unit 101 and the amount ofmovement of the focal point ΔBp here is given by Eq. (1).

As illustrated in FIGS. 3A and 3B, the amount of movement of the focuslens unit 101 required for moving the same amount of movement of thefocal point ΔBp is greater in FIG. 3A than in FIG. 3B, since the focussensitivity in FIG. 3A is lower than that in FIG. 3B. Thus, the amountof movement of the focus lens unit 101 per unit time can be reduced inthe case of FIG. 3B compared to the case of FIG. 3A. In other words, thesame moving speed of the focal point can be obtained by reducing thedriving speed of the focus lens unit 101. Thus, the driving speed of thefocus lens unit 101 to obtain a specific moving speed of the focal pointdepends on the imaging condition. In other words, the desirable movingspeed of the focus lens unit 101 varies depending on the imagingcondition.

(3) Power Consumption

The power consumption varies with the driving duration, the drivingspeed, and the driving acceleration of the focus lens unit 101.Specifically, the power consumption increases in a case where thedriving duration is long, the driving speed is high, or the drivingacceleration is high compared to a case where it is not. In other words,if the power consumption can be reduced by adaptation of the drivingperformance, imaging duration per single charging operation of a batterycan be increased or the battery can be miniaturized, for example, sincea battery capacity can be effectively used.

(4) Quietness

The driving of the focus lens unit 101 produces driving noise due tovibrations and friction. The driving noise varies with the driving speedand the driving acceleration of the focus lens unit 101. Specifically,the driving noise increases in a case where the driving speed is high orthe driving acceleration is high, compared to a case where it is not.The longer the focus lens unit 101 remains at rest, the more beneficialthe focus control can be in terms of quietness. Unpleasant driving noisecan be recorded during imaging in a quiet place. Thus, a capability ofchanging the driving noise depending on an imaging environment (ambientsound level) may be required.

<Relationship of Positioning Accuracy with Driving Speed, PowerConsumption, and Quietness>

A relationship of the positioning accuracy with the driving speed, thepower consumption, and the quietness will be described with reference toFIGS. 4A and 4B. FIGS. 4A and 4B are diagrams illustrating therelationship of the positioning accuracy with the driving speed, thepower consumption, and the quietness. FIGS. 4A and 4B illustrate themovement of the focus lens unit 101 to continue focusing on a movingobject in a case where the depth of focus is large and in a case wherethe depth of focus is small, respectively. In FIGS. 4A and 4B, thehorizontal axis represents time, and the vertical axis represents theposition of the focus lens unit 101. The vertical axis indicates adirection toward the infinity upward, and a direction toward the closestdistance downward.

The target position G of the focus lens unit 101 represents the positionof the focus lens unit 101 when an image of the object is focused on theimage pickup element 201. The depths of focus in FIGS. 4A and 4B are2Faδ and 2Fbδ, respectively. In FIG. 4A, a position GalimI indicates theposition of the focus lens unit 101 where the focal point is located atthe boundary of the depth of focus 2Faδ on the infinity side, and aposition GalimM indicates the position of the focus lens unit 101 wherethe focal point is located at the boundary of the depth of focus 2Faδ onthe closest distance side, with reference to the target position G. InFIG. 4B, a position GblimI indicates the position of the focus lens unit101 where the focal point is located at the boundary of the depth offocus 2Fbδ on the infinity side, and a position GblimM indicates theposition of the focus lens unit 101 where the focal point is located atthe boundary of the depth of focus 2Fbδ on the closest distance side,with reference to the target position G. A position (locus) Ca in FIG.4A and a position (locus) Cb in FIG. 4B indicate the position of thefocus lens unit 101 controlled so that the object falls within thedepths of focus 2Faδ and 2Fbδ, respectively.

In FIG. 4A, the depth of focus 2Faδ is large, and the object is lesslikely to go out of focus due to the control of the focus lens unit 101.By contrast, in FIG. 4B, the depth of focus 2Fbδ is small, and the locusCb of the focus lens unit 101 is to be controlled to make the deviationfrom the target position G smaller than that in FIG. 4A. Morespecifically, while the object is maintained in focus both in FIGS. 4Aand 4B, the driving along the locus Ca in FIG. 4A can reduce the drivingamount and the driving speed compared to the driving along the locus Cbin FIG. 4B. In other words, under an imaging condition where thepositioning accuracy is low, the focus lens unit 101 can be controlledwith low speed, low power consumption, and low noise.

<Relationship of Driving Speed with Positioning Accuracy, PowerConsumption, and Quietness>

A relationship of the driving speed with the positioning accuracy, thepower consumption, and the quietness will be described with reference toFIGS. 5A and 5B. FIGS. 5A and 5B are diagrams illustrating therelationship of the driving speed with the positioning accuracy, thepower consumption, and the quietness. In FIGS. 5A and 5B, the horizontalaxis represents time, and the vertical axis represents the position ofthe focus lens unit 101. FIG. 5A illustrates the position Ca of thefocus lens unit 101 in a case where the focus lens unit 101 is drivenfrom the position Pa1 to the position Pa2 illustrated in FIG. 3A in timeT0 to T1. FIG. 5B illustrates the position Cb of the focus lens unit 101in a case where the focus lens unit 101 is driven from the position Pb1to the position Pb2 illustrated in FIG. 3B in time T0 to T1. Asillustrated in FIGS. 3A and 3B, the amount of movement of the focalpoint in the case where the focus lens unit 101 is moved from theposition Pa1 to the position Pa2 is the same as the amount of movementof the focal point in the case where the focus lens unit 101 is movedfrom the position Pb1 to the position Pb2. Gradients of the positions Caand Cb in FIGS. 5A and 5B correspond to the driving speeds of the focuslens unit 101.

As illustrated in FIGS. 5A and 5B, the driving speed of the focus lensunit 101 to obtain the same amount of movement of the focal point ΔBp intime T0 and T1 is higher in the case of the position Ca than in the caseof the position Cb. In addition, since the driving speed correspondingto the position Ca is higher than that corresponding to the position Cb,the position Ca takes a long time to stabilize after the focus lens unit101 reaches the target position Pa2. By contrast, since the drivingspeed corresponding to the position Cb is lower than that correspondingto the position Ca, the position Cb takes only a short time to stabilizeafter the focus lens unit 101 reaches the target position Pb2. In otherwords, the driving speed affects the positioning accuracy. The drivingacceleration of the focus lens unit 101 corresponding to the position Cais also high, and the power consumption and the driving noise are alsohigh, compared to those corresponding to the position Cb. In otherwords, under an imaging condition where the required driving speed islow, the focus lens unit 101 can be controlled with high positioningaccuracy, low power consumption, and low noise.

<Second Information about Lens Apparatus>

Next, the second information about the lens apparatus 100 will bedescribed. The second information is information influencing the drivingperformance of the focus lens unit 101. As described above, for the sakeof adaptation of the driving performance in the driving control of thefocus lens unit 101, the control signal (driving signal) is to begenerated based on the second information influencing the drivingperformance. The second information is determined by the determinationunit 122. The second information includes information about the depth offocus and the focus sensitivity, for example. The determination unit 122obtains the information about the depth of focus from information aboutthe f-number and information about the permissible circle of confusion.The determination unit 122 stores information (table) indicating arelationship of the focus sensitivity with the position of the focuslens unit 101 and the position of the zoom lens unit 102, and obtainsthe information about the focus sensitivity from the relationship,information about the position of the focus lens unit 101, andinformation about the position of the zoom lens unit 102. Generating thecontrol signal based on such second information can provide a lensapparatus a benefit in term of the adaptation (customization) of thedriving performance such as the positioning accuracy, driving speed,power consumption, and quietness. A machine learning algorithm forgenerating the control signal based on the second information will bedescribed below.

<Machine Learning Model>

A method for the AI control unit 121 to generate the control signal byusing a machine learning model will be described. The AI control unit121 includes a machine learning model and operates based on a machinelearning algorithm. The machine learning algorithm here is, but notlimited to, an NN based algorithm (also referred to as an NN algorithm).The AI control unit 121 makes reference to a feature amount to be inputto an NN stored in the storage unit 123 and weights assigned to inputsto the respective layers, and generates an output related to the controlsignal by the NN algorithm using the feature amount and the weightsobtained by the reference. A method for generating the machine learningmodel (weights) will be described below.

A concept representing an input and output structure of the machinelearning model according to the first exemplary embodiment will bedescribed with reference to FIG. 6 . FIG. 6 is a diagram illustratinginputs and an output of the NN. In FIG. 6 , an input X1 is informationabout the driving instruction output from the driving control unit 125.An input X2 is information about the position of the focus lens unit 101obtained from the detector 106. An input X3 is information about thedepth of focus serving as the second information. An input X4 isinformation about the focus sensitivity serving as the secondinformation. An output Y1 is information about the output related to thecontrol signal for the driving device 105. Thus, the output Y1 of thetrained machine learning model is generated based on the inputs X1 toX4. The AI control unit 121 generates the output Y1 as a control signalor generates a control signal based on the output Y1, and controls thedriving device 105 by using the control signal.

<Method for Generating Machine Learning Model (Weights)>

Next, the method for generating the machine learning model (weights)(training by the machine learning unit 221) will be described. Thecontrol unit 211 transmits an instruction related to execution ofmachine learning to the machine learning unit 221 based on theoperator's operation on the operation device 206. Receiving theinstruction, the machine learning unit 221 starts machine learning. Theprocedure of the machine learning by the machine learning unit 221 willbe described with reference to FIG. 7 . FIG. 7 is a flowchartillustrating the processing procedure of the machine learning.

In step S101 of FIG. 7 , the machine learning unit 221 initializes themachine learning model (weights). Specifically, the machine learningunit 221 outputs initial values of the weights to the control unit 211.The control unit 211 receives the initial values of the weights from themachine learning unit 221, and transmits the initial values of theweights to the lens apparatus 100 via the communication unit 212. Thedriving control unit 125 of the lens apparatus 100 receives the initialvalues of the weights via the communication unit 126, and stores theinitial values in the storage unit 123. Subsequently, in step S102, themachine learning unit 221 obtains log information. Specifically, themachine learning unit 221 requests the control unit 211 to obtain loginformation about the lens apparatus 100. Receiving the request, thecontrol unit 211 requests the log information from the lens apparatus100 via the communication unit 212. The driving control unit 125 of thelens apparatus 100 receives the request for the log information via thecommunication unit 126, and instructs the AI control unit 121 to drivethe focus lens unit 101. The AI control unit 121 receives theinstruction for driving, and generates a control signal for the drivingdevice 105 based on the machine learning model using the weights storedin the storage unit 123. The machine learning unit 221 stores apredetermined training driving pattern for driving the focus lens unit101 from a start position to a stop position, and generates controlsignals corresponding to the training driving pattern. A trainingdriving pattern determined based on an autofocus algorithm may be usedinstead of the predetermined training driving pattern. The drivingcontrol unit 125 receives the request for the log information via thecommunication unit 126, and requests the log storage unit 124 to outputthe log information. The log storage unit 124 receives the outputrequest, and transmits the log information about the lens apparatus 100during driving of the focus lens unit 101 to the camera main body 200via the driving control unit 125 and the communication unit 126. The loginformation is stored in the log storage unit 222.

In step S103, the machine learning unit 221 evaluates the drivingperformance of the focus lens unit 101. Specifically, the machinelearning unit 221 evaluates the driving performance of the focus lensunit 101 driven by using the control signal generated by the AI controlunit 121 based on reward information stored in the reward storage unit223 and the log information stored in the log storage unit 222. Detailsof the evaluation will be described below. In step S104, the machinelearning unit 221 updates the machine learning model (weights).Specifically, the machine learning unit 221 updates the machine learningmodel (weights) based on an evaluation value resulting from theevaluation (for example, so that the evaluation value is maximized). Theweights can be updated by, but not limited to, backpropagation. Thegenerated weights (machine learning model) are stored in the storageunit 123 by processing similar to the processing of step S101.

In step S105, the machine learning unit 221 determines whether to endthe machine learning. Specifically, for example, the machine learningunit 221 makes the determination based on whether the number of times oftraining (weight update) reaches a predetermined value, or whether theamount of change in the evaluation value of the driving performance isless than a predetermined value. If the machine learning unit 221determines to not end the machine learning (NO in step S105), theprocessing returns to step S101, and the machine learning unit 221continues the machine learning. If the machine learning unit 221determines to end the machine learning (YES in step S105), theprocessing ends. The machine learning unit 221 employs a machinelearning model of which the evaluation satisfies an acceptance condition(for example, the amount of change in the evaluation value of thedriving performance is less than a predetermined value). The machinelearning unit 221 does not employ a machine learning model thatsatisfies an end condition (for example, the number of times of trainingreaches a predetermined value) and of which the evaluation does notsatisfy the acceptance condition.

The machine learning algorithm can be deep learning that uses an NN andgenerates the weights assigned to the inputs to the layers by itself.Deep learning can even generate feature amounts by itself. The machinelearning algorithm is not limited to deep learning, and other algorithmsmay be used. Examples may include at least one of the following: thenearest neighborhood algorithm, Naïve Bayes algorithm, a decision tree,and a support vector machine. Any of such algorithms available can beapplied to the present exemplary embodiment as appropriate.

A GPU can perform parallel data processing with high efficiency, and isthus effective in performing repetitive training using a machinelearning model such as one in deep learning. Thus, a GPU can be used forthe processing by the machine learning unit 221 instead of or inaddition to a CPU. For example, a machine learning program including amachine learning model can be executed by cooperation of a CPU and aGPU.

<Log Information>

Next, the log information will be described. The log informationincludes information targeted for the evaluation of the drivingperformance of the focus lens unit 101. The log storage unit 124collects and stores input/output information about the machine learningmodel, such as the inputs X1 to X4 and the output Y1 illustrated in FIG.6 , in each operation period of the machine learning model. The logstorage unit 124 stores information about the power consumption of thedriving device 105 obtained by the processor 120 as the log information.The log storage unit 124 also stores information about the drivinginstruction input to the AI control unit 121 and the position of thefocus lens unit 101 detected by the detector 106 as the log information.The log storage unit 124 also stores information about the targetposition and the positioning accuracy of the focus lens unit 101obtained by the processor 120 as the log information. The log storageunit 124 also stores information about the driving speed and the drivingacceleration of the focus lens unit 101 obtained from the informationabout the position of the focus lens unit 101 as the log information.The log storage unit 124 transmits the stored log information to thecamera main body 200 via the driving control unit 125 and thecommunication unit 126. The control unit 211 of the camera main body 200receives the log information via the communication unit 212, and storesthe log information in the log storage unit 222.

<Reward Information and Evaluation of Driving Performance>

The reward information is information for evaluating the drivingperformance. The reward information includes information about boundaryvalues for determining ranges and information about rewards determinedfor the respective ranges in advance for each of the types of drivingperformance. The reward information will be described with reference toFIGS. 8A1 to 8D2. FIGS. 8A1 to 8D2 are diagrams illustrating examples ofthe reward information. FIGS. 8A1, 8B1, 8C1, and 8D1 illustrate arelationship between time and a reward in training a machine learningmodel for the positioning accuracy, the driving speed, the drivingacceleration, and the power consumption serving as the drivingperformance, respectively. The horizontal axes of the graphs representtime. The vertical axes of the graphs represent the driving performanceand the boundary values. FIGS. 8A2, 8B2, 8C2, and 8D2 illustrate datastructures of the reward information about the positioning accuracy, thedriving speed, the driving acceleration, and the power consumption,respectively. The data structures include data on the boundary valuesand data on the rewards in the respective ranges.

The machine learning model is trained so that the evaluation of thedriving performance improves. Thus, for example, if the intended drivingperformance is the positioning accuracy, the highest reward is assignedto the range including a position deviation of 0. A specific type ofdriving performance is assigned relatively high rewards and therebygiven priority over another type of driving performance. For example,the power consumption is assigned relatively high rewards and therebygiven priority over the positioning accuracy. In the present exemplaryembodiment, the reward information will be described to includeinformation with two boundary values and information with three rewards.

The vertical axis of FIG. 8A1 indicates the value of a positiondeviation E that is the difference between the target position and theactual position of the focus lens unit 101. The positive direction ofthe position deviation E corresponds to a case where the actual positionof the focus lens unit 101 is on the infinity side of the targetposition. The negative direction of the position deviation E correspondsto a case where the actual position is on the closest distance side ofthe target position. The higher the frequency that the positiondeviation E is close to 0 (the smaller the total sum of positiondeviations E) is, the higher the positioning accuracy of the focus lensunit 101 is. FIG. 8A2 illustrates reward information RE about thepositioning accuracy. The reward information RE includes a boundaryvalue E1 and a boundary value E2 of the position deviation E, and areward SE1, a reward SE2, and a reward SE3 obtainable in respectiveranges. A range where the position deviation E is E1×−1 to E1 will bereferred to as a range AE1. A range obtained by excluding the range AE1from a range where the position deviation E is E2×−1 to E2 will bereferred to as a range AE2. A range obtained by excluding the ranges AE1and AE2 from the entire range will be referred to as a range AE3. Asillustrated in FIG. 8A2, the ranges AE1, AE2, and AE3 are assigned therewards SE1, SE2, and SE3, respectively. The relationship in magnitudebetween the rewards is the reward SE1>the reward SE2>the reward SE3. Thecloser to 0 the position deviation E is, the higher reward is assigned.As illustrated in FIG. 8A1, position deviations E at times Tp1, Tp2, andTp3 belong to the ranges AE2, AE3, and AE1, respectively. Thus, therewards obtainable at the times Tp1, Tp2, and Tp3 are the rewards SE2,SE3, and SE1, respectively. Here, the boundary value E1 can have a valueof Fδ/2, and the boundary value E2 can have a value of Fδ, for example.In other words, the highest reward SE1 is obtained if the actualposition of the focus lens unit 101 has a deviation less than or equalto one half of the depth of focus from the target position (|E|≤Fδ/2).If the actual position of the focus lens unit 101 has a deviationgreater than one half of the depth of focus and up to the depth of focusfrom the target position (Fδ/2<|E|≤Fδ), the intermediate reward SE2 isobtained. If the actual position of the focus lens unit 101 has adeviation beyond the depth of focus from the target position (|E|>Fδ),the lowest reward SE3 is obtained.

The vertical axis of FIG. 8B1 indicates the value of a driving speed Vof the focus lens unit 101. The positive direction of the driving speedV represents the direction toward the infinity. The negative directionof the driving speed V represents the direction toward the closestdistance. The closer to 0 the driving speed V is, the lower the drivingnoise is. FIG. 8B2 illustrates reward information RV about the drivingspeed V. The reward information RV includes boundary values V1 and V2 ofthe driving speed V, and rewards SV1, SV2, and SV3 obtainable inrespective ranges. A range where the driving speed V is V1×−1 to V1 willbe referred to as a range AV1. A range obtained by excluding the rangeAV1 from a range where the driving speed V is V2×−1 to V2 will bereferred to as a range AV2. A range obtained by excluding the ranges AV1and AV2 from the entire range will be referred to as a range AV3. Asillustrated in FIG. 8B2, the ranges AV1, AV2, and AV3 are assigned therewards SV1, SV2, and SV3, respectively. The relationship in magnitudebetween the rewards is the reward SV1>the reward SV2>the reward SV3. Thecloser to 0 the driving speed V is, the higher reward is assigned. Asillustrated in FIG. 8B1, driving speeds V at times Tp1, Tp2, and Tp3belong to the ranges AV2, AV3, and AV1, respectively. Thus, the rewardsobtainable at the times Tp1, Tp2, and Tp3 are the rewards SV2, SV3, andSV1, respectively. Here, the boundary values V1 and V2 are set based onthe relationship between the driving speed V and the driving noise, forexample. By setting the rewards so that the obtainable reward increasesas the driving speed V decreases, a machine learning model taking intoaccount quietness can be obtained since the driving noise decreases asthe driving speed V decreases.

The vertical axis of FIG. 8C1 indicates the value of a drivingacceleration A of the focus lens unit 101. The positive direction of thedriving acceleration A represents the direction toward the infinity. Thenegative direction of the driving acceleration A represents thedirection toward the closest distance. The closer to 0 the drivingacceleration A is, the lower the driving noise is. FIG. 8C2 illustratesreward information RA about the driving acceleration A. The rewardinformation RA includes boundary values A1 and A2 of the drivingacceleration A, and rewards SA1, SA2, and SA3 obtainable in respectiveranges. A range where the driving acceleration A is A1×−1 to A1 will bereferred to as a range AA1. A range obtained by excluding the range AA1from a range of A2×−1 to A2 will be referred to as a range AA2. A rangeobtained by excluding the ranges AA1 and AA2 from the entire range willbe referred to as a range AA3. As illustrated in FIG. 8C2, the rangesAA1, AA2, and AA3 are assigned the rewards SA1, SA2, and SA3,respectively. The relationship in magnitude between the rewards is thereward SA1>the reward SA2>the reward SA3. The closer to 0 the drivingacceleration A is, the higher reward is assigned. As illustrated in FIG.8C1, driving accelerations A at times Tp1, Tp2, and Tp3 belong to theranges AA1, AA3, and AA2, respectively. Thus, the rewards obtainable atthe times Tp1, Tp2, and Tp3 are the rewards SA1, SA3, and SA2,respectively. Here, the boundary values A1 and A2 are set based on therelationship between the driving acceleration A and the driving noise,for example. By setting the rewards so that the obtainable rewardincreases as the driving acceleration A decreases, a machine learningmodel taking into account quietness can be obtained since the drivingnoise decreases as the driving acceleration A decreases.

The vertical axis of FIG. 8D1 indicates the value of power consumption Pof the driving device 105. FIG. 8D2 illustrates reward information RPabout the power consumption P. The reward information RP includesboundary values P1 and P2 of the power consumption P, and rewards SP1,SP2 and SP3 obtainable in respective ranges. A range where the powerconsumption P is 0 to P1 will be referred to as a range AP1. A rangewhere the power consumption P is higher than P1 and not higher than P2will be referred to as a range AP2. A range obtained by excluding theranges AP1 and AP2 from the entire range will be referred to as a rangeAP3. As illustrated in FIG. 8D2, the ranges AP1, AP2, and AP3 areassigned the rewards SP1, SP2, and SP3, respectively. The relationshipin magnitude between the rewards is the reward SP1>the reward SP2>thereward SP3. The closer to 0 the power consumption P is, the higherreward is assigned. As illustrated in FIG. 8D1, power consumptions P attimes Tp1, Tp2, and Tp3 belong to the ranges AP1, AP3, and AP2,respectively. Thus, the rewards obtainable at the times Tp1, Tp2 and Tp3are the rewards SP1, SP3, and SP2, respectively. By setting the rewardsso that the obtainable reward increases as the power consumptiondecreases, a machine learning model taking into account low powerconsumption can be obtained.

In such a manner, the reward information for evaluating the drivingperformance such as the positioning accuracy (position deviation), thedriving speed, the driving acceleration, and the power consumption canbe set. Using the reward information, the machine learning unit 221 cangenerate rewards for the respective types of driving performance in eachunit time based on the log information in driving the focus lens unit101, and accumulate the rewards to evaluate the machine learning model.Being based on the rewards related to a plurality of types of drivingperformance is beneficial in customizing the machine learning model. Thepower consumption may be measured based on the current flowing throughthe driving device 105, or estimated based on the driving speed and/orthe driving acceleration. The boundary values are not limited toconstant ones and can be changed as appropriate. The rewards are notlimited to ones determined based on the boundary values, and may bedetermined based on functions related to the respective types of drivingperformance. In such a case, the reward information can includeinformation about the functions.

<First Reward Section and Second Reward Section>

Next, a first reward section and a second reward section of the rewardinformation will be described. FIG. 9 is a diagram illustrating a datastructure of the reward information. Information about the first rewardsection (first reward information prepared in advance) includesinformation about a reward REb related to the positioning accuracy, areward RVb related to the driving speed, a reward RAb related to thedriving acceleration, and a reward RPb related to the power consumption.Information about the second reward section (second reward information)includes information about a reward REu related to the positioningaccuracy, a reward RVu related to the driving speed, a reward RAurelated to the driving acceleration, and a reward RPu related to thepower consumption. The rewards REb and REu have a data structure similarto that of the reward information RE about the positioning accuracyillustrated in FIG. 8A2. The rewards RVb and RVu have a data structuresimilar to that of the reward information RV about the driving speedillustrated in FIG. 8B2. The rewards RAb and RAu have a data structuresimilar to that of the reward information RA about the drivingacceleration illustrated in FIG. 8C2. The rewards RPb and RPu have adata structure similar to that of the reward information RP about thepower consumption illustrated in FIG. 8D2.

The information about the first reward section is information aboutrewards specific to the lens apparatus 100. The information about thefirst reward section is stored in the first reward section storage unit224 in advance as reward information specific to the lens apparatus 100.The information about the second reward section is information aboutrewards that are variable based on a request from the operator of thelens apparatus 100. The information about the second reward section isstored in the second reward section storage unit 225 based on theoperator's request. The reward storage unit 223 stores the informationabout the first reward section and the information about the secondreward section.

The information about the first reward section is reward information forobtaining allowable driving performance of the lens apparatus 100, andthus includes wider ranges of reward settings including negative valuesthan the information about the second reward section does. Theinformation about the second reward section is variable based on theoperator's request, and can be obtained based on information about therequest and information about options for the second reward section. Thereward information is obtained from the information about the firstreward section and the information about the second reward section. Amachine learning model is trained (generated) by obtaining theevaluation value of the machine learning model based on the rewardinformation as described with reference to FIGS. 8A1 to 8D2.

A method for obtaining the information about the second reward sectionbased on the operator's request will now be described. FIGS. 10A to 10Care diagrams illustrating a data structure of the information about theoptions for the second reward section. FIG. 10A illustrates a datastructure of information about an option UREu for the second rewardsection related to the positioning accuracy. The information about theoption UREu includes boundary values of the position deviation andreward information about respective ranges defined by the boundaryvalues at each level. FIG. 10B illustrates a data structure ofinformation about an option URSu for the second reward section relatedto the quietness. The information about the option URSu includesinformation about an option URVu for the second reward section relatedto the driving speed and information about an option URAu for the secondreward section related to the driving acceleration. The informationabout the option URVu includes boundary values of the driving speed andreward information about respective ranges defined by the boundaryvalues at each level. The information about the option URAu includesboundary values of the driving acceleration and reward information aboutrespective ranges defined by the boundary values at each level. FIG. 10Cillustrates a data structure of information about an option URPu for thesecond reward section related to the power consumption. The informationabout the options URPu includes boundary values of the power consumptionand reward information about respective ranges defined by the boundaryvalues at each level.

The information about the option UREu for the second reward sectionrelated to the positioning accuracy, the information about the optionURSu for the second reward section related to the quietness, and theinformation about the option URPu for the second reward section relatedto the power consumption are set in the following manner. In each ofthese types of information, the boundary values and reward values areset so that the operator's request level decreases in order (ascendingorder) of levels 1, 2, and 3. More specifically, for example, theboundary values at level 1 are close to the target value of the drivingperformance and the reward values are high, compared to those at theother levels.

The operator's request can be input via the operation device 206illustrated in FIG. 1 . Based on the request, the level of each type ofdriving performance can be selected from levels 1 to 3. Informationabout the level is transmitted to the second reward section storage unit225 via the control unit 211. The second reward section storage unit 225identifies (selects) information about the second reward section relatedto each type of driving performance based on the information about thelevel of each type of driving performance. Thus, a customized machinelearning model (weights) can be generated by training the machinelearning model (weights) based on the customized information about therewards. The information about the generated machine learning model(weights) is transmitted from the camera main body 200 to the lensapparatus 100, stored in the storage unit 123, and used to control thedriving (driving device 105) of the focus lens unit 101.

Other Examples of Object to be Controlled

While the driving control is described to be targeted for the focus lensunit 101, the present exemplary embodiment is not limited thereto. Inthe present exemplary embodiment, the driving control may be targetedfor other optical members such as the zoom lens unit 102, the imagestabilization lens unit 104, a flange back adjustment lens unit, and theaperture stop 103. Positioning accuracy, quietness, and powerconsumption are the driving performance also to be taken into account indriving such optical members. The required positioning accuracy of thezoom lens unit 102 can vary depending on the relationship between thedriving amount and the amount of change in the angle of view or the sizeof the object. The required positioning accuracy of the imagestabilization lens unit 104 can vary with the focal length. The requiredpositioning accuracy of the aperture stop 103 can vary depending on therelationship between the driving amount and the amount of change in theluminance of the video image.

Other Examples of Second Information

The information about the focus sensitivity and the depth of focus hasbeen described to be the second information about the lens apparatus100. However, this is not restrictive, and the second information mayinclude information about at least one of the orientation, temperature,and ambient sound level of the lens apparatus 100. Depending on theorientation of the lens apparatus 100, the effect of the gravity on theoptical members is changed, whereby the load (torque) of the drivingdevice 105 can be changed. Depending on the temperature of the lensapparatus 100, the property of a lubricant in the driving system ischanged, whereby the load (torque) of the driving device 105 can bechanged. The sound level around the lens apparatus 100 influences theconstraints on the driving noise of the driving device 105, whereby thelimitations on the speed and acceleration of the driving device 105 canbe changed.

As described above, in the present exemplary embodiment, for example, alens apparatus or an image pickup apparatus beneficial in terms ofadaptation (customization) of the driving performance can be provided.

Second Exemplary Embodiment

«Configuration Example Where Lens Apparatus Includes Training Unit(Generator)»

A second exemplary embodiment will be described with reference to FIGS.11 to 15B. FIG. 11 is a diagram illustrating a configuration example ofa lens apparatus according to the second exemplary embodiment, and byextension, is a diagram illustrating a configuration example of a system(image pickup apparatus) including a configuration example of a cameramain body as well. The system is different from that of the firstexemplary embodiment in that a lens apparatus 100 includes a trainingunit. Another difference from the first exemplary embodiment is thatsecond information about the lens apparatus 100 includes informationabout recording by the camera main body.

A training unit 1220 can include a processor (such as a CPU or a GPU)and a storage device (such as a ROM, RAM, or HDD). The training unit1220 can include a machine learning unit 1221, a log storage unit 1222,a reward storage unit 1223, a first reward section storage unit 1224,and a second reward section storage unit 1225. The training unit 1220also stores a program for controlling operation of these units.

A driving control unit 1125 has a function of exchanging informationwith the training unit 1220 in addition to the functions of the drivingcontrol unit 125 according to the first exemplary embodiment. An AIcontrol unit 1121 controls driving (driving device 105) of a focus lensunit 101 based on a machine learning model generated by the trainingunit 1220. A determination unit 1122 is a determination unit thatdetermines information (second information) about the lens apparatus 100for the AI control unit 1121 to use. The second information will bedescribed below. An operation device 1206 is an operation device for theoperator to operate the lens apparatus 100 (image pickup apparatus).

<Second Information>

The second information here includes information about the effects ofthe driving control of the focus lens unit 101 on recording by a cameramain body 200. In the present exemplary embodiment, the driving of thefocus lens unit 101 can be controlled by taking into account the effectsof the control on the recording, based on such second information inaddition to or instead of the second information according to the firstexemplary embodiment. The second information can include informationthat is obtained by a control unit 211 analyzing image data obtained bya signal processing circuit 203. The second information can bedetermined based on information transmitted from the control unit 211 tothe determination unit 1122 via a communication unit 212, acommunication unit 126, and the driving control unit 1125. For example,the second information can be information about at least one of thefollowing: the permissible circle of confusion, the defocus amount ofthe object obtained by imaging by the camera main body 200, and a soundlevel (level of recorded ambient sound) obtained by a microphoneincluded in the camera main body 200. The determination unit 1122 canobtain information about the depth of focus from information about anf-number and the permissible circle of confusion.

<Machine Learning Model>

A machine learning model in the AI control unit 1121 will now bedescribed. FIG. 12 is a diagram illustrating inputs and an output of anNN. In the NN according to the second exemplary embodiment illustratedin FIG. 12 , an input X21 is information about a driving instructionoutput from the driving control unit 1125. An input X22 is informationabout the position of the focus lens unit 101 obtained from a detector106. An input X23 is information about the depth of focus obtained asthe second information as describe above. An input X24 is informationabout the focus sensitivity serving as the second information. An inputX25 is information about the defocus amount of the object obtained asthe second information as described above. An input X26 is informationabout the sound level obtained as the second information as describedabove. An output Y21 is information about an output related to a controlsignal for the driving device 105. In such a manner, the output Y21 ofthe trained machine learning model is generated based on the inputs X21to X26. The AI control unit 1121 generates the output Y21 as a controlsignal or generates a control signal based on the output Y21, andcontrols the driving device 105 by using the control signal.

<Log Information>

Log information according to the second exemplary embodiment will bedescribed. A log storage unit 1124 collects and stores input/outputinformation about the machine learning model, such as the inputs X21 toX26 and the output Y21 illustrated in FIG. 12 , in each operation periodof the machine learning model. The log storage unit 1124 storesinformation about the power consumption of the driving device 105obtained by a processor 120 as the log information. The log storage unit1124 also stores information about the driving instruction input to theAI control unit 1121 and the position of the focus lens unit 101detected by the detector 106 as the log information. The log storageunit 1124 also stores information about the target position and thepositioning accuracy of the focus lens unit 101 obtained by theprocessor 120 as the log information. The log storage unit 1124 alsostores information about the driving speed and the driving accelerationof the focus lens unit 101 obtained from the information about theposition of the focus lens unit 101 as the log information. The logstorage unit 1124 also stores information indicating a relationshipbetween at least one of the driving speed and driving acceleration and adriving noise level, and stores information about the driving noiselevel generated based on information about the at least one of thedriving speed and driving acceleration and the information indicatingthe relationship. The log storage unit 1124 also obtains a ratio of arecording sound level to the driving noise level (signal-to-noise (S/N)ratio with the driving noise as the noise), and stores information aboutthe ratio. The S/N ratio indicates the effect of the driving noise onrecording. The higher the S/N ratio, the smaller the effect of thedriving noise on recording. The log storage unit 1124 stores the storedlog information into the log storage unit 1222 via the driving controlunit 1125.

<Reward Information and Evaluation of Driving Performance>

Reward information according to the second exemplary embodiment will bedescribed with reference to FIGS. 13A1 to 13B2. FIGS. 13A1 to 13B2 arediagrams illustrating the reward information. FIGS. 13A1 and 13B1illustrate a relationship between time and a reward in training themachine learning model with respect to the defocus amount and the S/Nratio serving as driving performance, respectively. The horizontal axesof the graphs of FIGS. 13A1 and 13B1 represent time. FIGS. 13A2 and 13B2illustrate a data structure of reward information with respect to thedefocus amount and the S/N ratio, respectively. Similar to the datastructure in the first exemplary embodiment, the data structure includesdata on boundary values and data on rewards in respective ranges definedby the boundary values with respect to each type of driving performance.

The vertical axis of FIG. 13A1 indicates the value of a defocus amountD. The defocus amount D has a positive value if the focal point is offto the infinity side and a negative value if the focal point is off tothe closest distance side. FIG. 13A2 illustrates reward information RDabout the defocus amount D. The reward information RD includes aboundary value D1 and a boundary value D2 of the defocus amount D, and areward SD1, a reward SD2, and a reward SD3 obtainable in respectiveranges. A range where the defocus amount D is D1×−1 to D1 will bereferred to as a range AD1. A range obtained by excluding the range AD1from a range of D2×−1 to D2 will be referred to as a range AD2. A rangeobtained by excluding the ranges AD1 and AD2 from the entire range willbe referred to as a range AD3. As illustrated in FIG. 13A2, the rangesAD1, AD2, and AD3 are assigned the rewards SD1, SD2, and SD3,respectively. The relationship in magnitude between the rewards is thereward SD1>the reward SD2>the reward SD3. The closer to 0 the defocusamount D is, the higher reward is assigned. As illustrated in FIG. 13A1,defocus amounts D at times Tp1, Tp2, and Tp3 belong to the ranges AD2,AD3, and AD1, respectively. Thus, the rewards obtainable at the timesTp1, Tp2, and Tp3 are the rewards SD2, SD3, and SD1, respectively. Here,the boundary value D1 can have a value of Fδ/2, and the boundary valueD2 can have a value of Fδ, for example. In other words, the highestreward SD1 is obtained if the defocus amount D has a value less than orequal to one half of the depth of focus (|D|≤Fδ/2). If the defocusamount D has a value greater than one half of the depth of focus and upto the depth of focus (Fδ/2<|D|≤Fδ), the intermediate reward SD2 isobtained. If the defocus amount D has a value exceeding the depth offocus (|D|>Fδ), the lowest reward SD3 is obtained.

The vertical axis of FIG. 13B1 indicates the value of an S/N ratio N.The higher the S/N ratio N, the smaller the effect of the driving noiseon recording quality. FIG. 13B2 illustrates reward information RN aboutthe S/N ratio. The reward information RN includes a boundary value N1and a boundary value N2 of the S/N ratio, and a reward SN1, a rewardSN2, and a reward SN3 obtainable in respective ranges. A range where theS/N ratio is 0 to N1 will be referred to as a range AN1. A range of N1to N2 will be referred to as a range AN2. A range obtained by excludingthe ranges AN1 and AN2 from the entire range will be referred to as arange AN3. As illustrated in FIG. 13B2, the ranges AN1, AN2, and AN3 areassigned the rewards SN1, SN2, and SN3, respectively. The relationshipin magnitude between the rewards is the reward SN1<the reward SN2<thereward SN3. The closer to 0 the S/N ratio N is, the lower reward isassigned. As illustrated in FIG. 13B1, S/N ratios N at times Tp1, Tp2,and Tp3 belong to the ranges AN1, AN3, and AN2, respectively. Thus, therewards obtainable at the times Tp1, Tp2, and Tp3 are the rewards SN1,SN3, and SN2, respectively. Since the rewards are set so that theobtainable reward increases as the S/N ratio increases, a machinelearning model beneficial in terms of recording quality can begenerated.

The reward information for evaluating the defocus amount serving as thedriving performance and the S/N ratio related to driving noise can beset as described above. Using such reward information, the machinelearning unit 1221 can generate rewards for the respective types ofdriving performance in each unit time based on the log information indriving the focus lens unit 101, and accumulate the rewards to evaluatethe machine learning model. Being based on the rewards related to aplurality of types of driving performance is beneficial in customizingthe machine learning model. The boundary values are not limited toconstant ones and can be changed as appropriate. The rewards are notlimited to ones determined based on the boundary values, and may bedetermined based on functions related to the respective types of drivingperformance. In such a case, the reward information can includeinformation about the functions.

<First Reward Section and Second Reward Section>

Next, information about a first reward section and information about asecond reward section according to the present exemplary embodiment willbe described. FIG. 14 is a diagram illustrating a data structure of thereward information. The information about the first reward sectionincludes information about a reward RDb related to the defocus amountand a reward RNb related to the S/N ratio. The information about thesecond reward section includes information about a reward RDu related tothe defocus amount and a reward RNu related to the S/N ratio. Therewards RDb and RDu have a data structure similar to that of the rewardinformation RD about the defocus amount illustrated in FIG. 13A2. Therewards RNb and RNu have a data structure similar to that of the rewardinformation RN about the S/N ratio illustrated in FIG. 13B2.

The information about the first reward section is information aboutrewards specific to the lens apparatus 100. The information about thefirst reward section is stored in the first reward section storage unit1224 in advance as reward information specific to the lens apparatus100. The information about the second reward section is informationabout rewards variable based on a request from the operator of the lensapparatus 100. The information about the second reward section is storedin the second reward section storage unit 1225 based on the operator'srequest. The reward storage unit 1223 stores the information about thefirst reward section and the information about the second rewardsection.

The information about the first reward section is reward information forobtaining allowable driving performance of the lens apparatus 100, andthus includes wider ranges of reward settings including negative valuesthan the information about the second reward section does. Theinformation about the second reward section is variable based on theoperator's request, and can be obtained based on information about therequest and information about options for the second reward section. Thereward information is obtained from the information about the firstreward section and the information about the second reward section. Amachine learning model is trained (generated) by obtaining theevaluation value of the machine learning model based on the rewardinformation as described with reference to FIGS. 13A1 to 13B2.

A method for obtaining the information about the second reward sectionbased on the operator's request will now be described. FIGS. 15A and 15Bare diagrams illustrating a data structure of the information about theoptions for the second reward section. FIG. 15A illustrates a datastructure of information about an option URDu for the second rewardsection related to the defocus amount. The information about the optionURDu includes boundary values of the defocus amount and rewardinformation about respective ranges defined by the boundary values ateach level. FIG. 15B illustrates a data structure of information aboutan option URNu for the second reward section related to the quietness(S/N ratio). The information about the option URNu includes boundaryvalues of the S/N ratio and reward information about respective rangesdefined by the boundary values at each level.

In both the information about the option URDu for the second rewardsection related to the defocus amount and the information about theoption URNu for the second reward section related to the quietness (S/Nratio), the boundary values and the reward values are set so that theoperator's request level decreases in order (ascending order) of levels1, 2, and 3. More specifically, for example, the boundary values atlevel 1 are close to the target value of the driving performance and thereward values are high, compared to those at the other levels.

The operator's request can be input via the operation device 1206illustrated in FIG. 11 . Based on the request, the level of each type ofdriving performance can be selected from levels 1 to 3. Informationabout the level is transmitted to the second reward section storage unit1225 via the driving control unit 1125. The second reward sectionstorage unit 1225 identifies (selects) information about the secondreward section related to each type of driving performance based on theinformation about the level of each type of driving performance. Thus, acustomized machine learning model (weights) can be generated by trainingthe machine learning model (weights) based on the customized informationabout the rewards. The information about the generated machine learningmodel (weights) is transmitted from the machine learning unit 1221,stored in the storage unit 123, and used to control the driving (drivingdevice 105) of the focus lens unit 101.

Other Examples of Object to be Controlled

While the driving control is described to be targeted for the focus lensunit 101, the present exemplary embodiment is not limited thereto. Inthe present exemplary embodiment, the driving control may be targetedfor other optical members such as a zoom lens unit, an imagestabilization lens unit, a flange back adjustment lens unit, and anaperture stop. A defocus amount and quietness (S/N ratio) are thedriving performance also to be taken into account in driving suchoptical members. If such other optical members are subjected to thedriving control, information about other types of driving performancemay be taken into account as the second information in addition to orinstead of the defocus amount.

As described above, in the present exemplary embodiment, for example, alens apparatus or an image pickup apparatus beneficial in terms ofadaptation (customization) of the driving performance can be provided.

Third Exemplary Embodiment

«Configuration Example Where Remote Apparatus (Processing Apparatus)Includes Training Unit (Generator)»

A third exemplary embodiment will be described with reference to FIG. 16. FIG. 16 is a diagram illustrating a configuration example of a lensapparatus according to the third exemplary embodiment, and by extension,is a diagram illustrating a configuration example of a system (imagepickup apparatus) including a configuration example of a camera mainbody as well. The system is different from that of the first exemplaryembodiment in that a remote apparatus 400 is included and the remoteapparatus 400 includes a training unit. A camera main body 200 includesa communication unit 230 for communicating with the remote apparatus400. The remote apparatus 400 can be a processing apparatus such as amobile terminal or a computer terminal, for example. The remoteapparatus 400 includes a display unit 401, an operation device 402, aprocessor 410, and a training unit 420. The processor 410 includes acontrol unit 411 and a communication unit 412. The communication unit412 is used to communicate with the camera main body 200. Thecommunication unit 412 and the communication unit 230 of the camera mainbody 200 communicate wirelessly although the communication method is notlimited to wireless communication. The wireless communication can beknown wireless communication over a wireless local area network (LAN).

The training unit 420 can include a processor (such as a CPU or a GPU)and a storage device (such as a ROM, RAM, or HDD). The training unit 420can include a machine learning unit 421, a log storage unit 422, areward storage unit 423, a first reward section storage unit 424, and asecond reward section storage unit 425. The training unit 420 alsostores a program for controlling operation of these units. The trainingunit 420 can make an operation similar to that of the training unit 220according to the first exemplary embodiment.

In the present exemplary embodiment, unlike the first exemplaryembodiment, the training unit is not included in the camera main body200 but in the remote apparatus 400. Thus, information transmissionbetween a processor 210 of the camera main body 200 and the trainingunit 420 is performed via the communication unit 230, the communicationunit 412, and the control unit 411. Image data output from a signalprocessing circuit 203 is transmitted to the control unit 411 via acontrol unit 211, the communication unit 230, and the communication unit412. The image data transmitted to the control unit 411 is displayed onthe display unit 401.

The control unit 411 can transmit an instruction related to execution ofmachine learning to the machine learning unit 421 based on theoperator's operation on the operation device 402. The control unit 211can transmit an instruction related to the execution of machine learningto the machine learning unit 421 via the control unit 411 based on theoperator's operation on an operation device 206. Receiving theinstruction, the machine learning unit 421 starts machine learning.Similarly, information about the level of the second reward sectionrelated to each type of driving performance, input by the operator fromthe operation device 402 or the operation device 206, is transmitted tothe second reward section storage unit 425 via the control unit 411. Thesecond reward section storage unit 425 identifies (selects) informationabout the second reward section related to each type of drivingperformance based on the information about the level of each type ofdriving performance. Thus, a customized machine learning model (weights)can be generated by training the machine learning model (weights) basedon the customized information about the rewards. The information aboutthe generated machine learning model (weights) is transmitted from theremote apparatus 400 to the lens apparatus 100, stored in a storage unit123, and used to control the driving (driving device 105) of a focuslens unit 101.

In such a manner, a customized machine learning model can be generatedat a remote location away from the lens apparatus 100 in a state wherean image obtained by the camera main body 200 can be observed (watched).The camera main body 200 may issue an instruction for executing machinelearning and an instruction for setting the second reward section viathe operation device 206 while the remote apparatus 400 performs onlythe machine learning processing that requires high-speed calculationprocessing.

As described above, in the present exemplary embodiment, for example, alens apparatus, an image pickup apparatus, or a processing apparatusbeneficial in terms of adaptation (customization) of driving performancecan be provided.

In the first and third exemplary embodiments, the second informationabout the lens apparatus 100 to be used to train the machine learningmodel is described to be only information specific to the lens apparatus100. In the second exemplary embodiment, the second information isdescribed to include both the information specific to the lens apparatus100 and information specific to the camera main body 200. However, thisis not restrictive. The second information may include only theinformation specific to the camera main body 200.

Exemplary Embodiments Related to Program, Storage Medium, and DataStructure

An exemplary embodiment of the disclosure can be implemented bysupplying a program or data (structure) for implementing one or morefunctions or methods of the foregoing exemplary embodiments to a systemor an apparatus via a network or a storage medium. In such a case, acomputer in the system or the apparatus can read the program or the data(structure) and perform processing based on the program or the data(structure). The computer can include one or a plurality of processorsor circuits, and can include a network including a plurality of separatecomputers or a plurality of separate processors or circuits, to read andexecute computer-executable instructions.

The processor(s) or circuit(s) can include a CPU, a microprocessing unit(MPU), a GPU, an application specific integrated circuit (ASIC), or afield programmable gate array (FPGA). The processor(s) or circuit(s) canalso include a digital signal processor (DSP), a data flow processor(DFP), or a neural processing unit (NPU).

While the exemplary embodiments of the disclosure have been describedabove, it will be understood that the disclosure is not limited to theexemplary embodiments, and various modifications and changes may be madewithout departing from the gist thereof.

Other Embodiments

Embodiment(s) of the disclosure can also be realized by a computer of asystem or apparatus that reads out and executes computer executableinstructions (e.g., one or more programs) recorded on a storage medium(which may also be referred to more fully as a ‘non-transitorycomputer-readable storage medium’) to perform the functions of one ormore of the above-described embodiment(s) and/or that includes one ormore circuits (e.g., application specific integrated circuit (ASIC)) forperforming the functions of one or more of the above-describedembodiment(s), and by a method performed by the computer of the systemor apparatus by, for example, reading out and executing the computerexecutable instructions from the storage medium to perform the functionsof one or more of the above-described embodiment(s) and/or controllingthe one or more circuits to perform the functions of one or more of theabove-described embodiment(s). The computer may comprise one or moreprocessors (e.g., central processing unit (CPU), micro processing unit(MPU)) and may include a network of separate computers or separateprocessors to read out and execute the computer executable instructions.The computer executable instructions may be provided to the computer,for example, from a network or the storage medium. The storage mediummay include, for example, one or more of a hard disk, a random-accessmemory (RAM), a read only memory (ROM), a storage of distributedcomputing systems, an optical disk (such as a compact disc (CD), digitalversatile disc (DVD), or Blu-ray Disc (BD)™), a flash memory device, amemory card, and the like.

While the disclosure has been described with reference to exemplaryembodiments, it is to be understood that the disclosure is not limitedto the disclosed exemplary embodiments. The scope of the followingclaims is to be accorded the broadest interpretation so as to encompassall such modifications and equivalent structures and functions.

This application claims the benefit of Japanese Patent Application No.2020-033351, filed Feb. 28, 2020, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. A lens apparatus comprising: an optical member; adriving device including a motor and configured to perform driving ofthe optical member; a detector configured to detect a state related tothe driving; and a processor configured to generate a control signal forthe driving device based on first information about the detected state,the processor including a machine learning model configured to generatean output related to the control signal based on the first informationand second information about a state of the lens apparatus related todriving performance of the driving device; an operation device for anoperator to input information about a requirement for the drivingperformance of the driving device, and a generator including aprocessing unit and configured to obtain information about a reward forevaluating the machine learning model based on the information about therequirement, and perform generation of the machine learning model basedon the first and second information and the information about thereward, the evaluating being used to employ the machine learning model,wherein the generator is configured to make an evaluation of the machinelearning model based on the information about the reward, and isconfigured not to employ the machine learning model that satisfies anend condition and of which the evaluation does not satisfy an acceptancecondition.
 2. The lens apparatus according to claim 1, wherein the inputinformation about the requirement relates to each of a plurality oftypes of the driving performance.
 3. The lens apparatus according toclaim 1, wherein the generator includes previously-prepared informationabout a first reward, and is configured to perform the generation of themachine learning model based on the information about the first rewardand information about a second reward, the information about the secondreward being obtained based on the input information about therequirement.
 4. The lens apparatus according to claim 1, wherein thegenerator is configured to obtain the information about the reward basedon the second information.
 5. The lens apparatus according to claim 4,wherein the optical member is a lens unit configured to change an objectdistance, and wherein the second information includes information aboutat least one of a depth of focus of the lens apparatus, an amount ofmovement of a focal point of the lens apparatus per a unit amount ofmovement of the lens unit, and sound around the lens apparatus.
 6. Thelens apparatus according to claim 1, wherein the generator is configuredto obtain the information about the reward based on information from animage pickup apparatus main body on which the lens apparatus is mounted.7. The lens apparatus according to claim 6, wherein the information fromthe image pickup apparatus main body includes information about at leastone of a diameter of a permissible circle of confusion, a defocusamount, and a sound level.
 8. The lens apparatus according to claim 6,wherein the processor is configured to generate the control signal basedon the information from the image pickup apparatus main body.
 9. Thelens apparatus according to claim 1, wherein the generator is configuredto employ the machine learning model of which the evaluation satisfiesthe acceptance condition.
 10. An image pickup apparatus comprising: thelens apparatus according to claim 1; and an image pickup elementincluding an image sensor and configured to pick up an image formed bythe lens apparatus.
 11. A processing apparatus configured to performprocessing related to a machine learning model in a lens apparatus, thelens apparatus including an optical member, a driving device including amotor and configured to perform driving of the optical member, adetector configured to detect a state related to the driving, and aprocessor configured to generate a control signal for the driving devicebased on first information about the detected state, the processorincluding a machine learning model configured to generate an outputrelated to the control signal based on the first information and secondinformation about a state of the lens apparatus related to drivingperformance of the driving device, the processor being configured tooutput the first information and the second information to a generatorincluding a processing unit and configured to perform generation of themachine learning model, the processing apparatus comprising: anoperation device for an operator to input information about arequirement for the driving performance of the driving device, whereinthe processing apparatus includes a processing unit and is configured toobtain information about a reward for evaluating the machine learningmodel based on the information about the requirement, the generatorbeing configured to perform the generation of the machine learning modelbased on the first and second information and the information about thereward, the evaluating being used to employ the machine learning model,the generator being configured to make an evaluation of the machinelearning model based on the information about the reward, and beingconfigured not to employ the machine learning model that satisfies anend condition and of which the evaluation does not satisfy an acceptancecondition.
 12. The processing apparatus according to claim 11, furthercomprising the generator.
 13. The processing apparatus according toclaim 12, wherein the generator includes previously-prepared informationabout a first reward, and is configured to perform the generation of themachine learning model based on the information about the first rewardand information about a second reward, the information about the secondreward being obtained based on the input information about therequirement.
 14. The processing apparatus according to claim 12, whereinthe generator is configured to obtain the information about the rewardbased on the second information.
 15. An image pickup apparatus main bodyon which a lens apparatus is mounted, the image pickup apparatus mainbody comprising the processing apparatus according to claim
 11. 16. Animage pickup apparatus main body on which a lens apparatus is mounted,the image pickup apparatus main body comprising the processing apparatusaccording to claim
 12. 17. A processing method of performing processingrelated to a machine learning model in a lens apparatus, the lensapparatus including an optical member, a driving device including amotor and configured to perform driving of the optical member, adetector configured to detect a state related to the driving, and aprocessor configured to generate a control signal for the driving devicebased on first information about the detected state, the processorincluding a machine learning model configured to generate an outputrelated to the control signal based on the first information and secondinformation about a state of the lens apparatus related to drivingperformance of the driving device, the processor being configured tooutput the first information and the second information to a generatorincluding a processing unit and configured to perform generation of themachine learning model, the processing method comprising: obtaininginformation about a reward for evaluating the machine learning modelbased on information about a requirement for the driving performance ofthe driving device, the information about the requirement being inputfrom an operation device operated by an operator, the generator beingconfigured to perform generation of the machine learning model based onthe first and second information and the information about the reward,the evaluating being used to employ the machine learning model, thegenerator being configured to make an evaluation of the machine learningmodel based on the information about the reward, and being configurednot to employ the machine learning model that satisfies an end conditionand of which the evaluation does not satisfy an acceptance condition.18. The processing method according to claim 17, further comprisinggenerating the machine learning model by the generator based on theinformation about the reward.
 19. A non-transitory computer-readablestorage medium storing a program for causing a computer to perform aprocessing method for performing processing related to a machinelearning model in a lens apparatus, the lens apparatus including anoptical member, a driving device including a motor and configured toperform driving of the optical member, a detector configured to detect astate related to the driving, and a processor configured to generate acontrol signal for the driving device based on first information aboutthe detected state, the processor including a machine learning modelconfigured to generate an output related to the control signal based onthe first information and second information about a state of the lensapparatus related to driving performance of the driving device, theprocessor being configured to output the first information and thesecond information to a generator including a processing unit andconfigured to perform generation of the machine learning model, theprocessing method comprising: obtaining information about a reward forevaluating the machine learning model based on information about arequirement for the driving performance of the driving device, theinformation about the requirement being input from an operation deviceoperated by an operator, the generator being configured to performgeneration of the machine learning model based on the first and secondinformation and the information about the reward, the evaluating beingused to employ the machine learning model, the generator beingconfigured to make an evaluation of the machine learning model based onthe information about the reward, and being configured not to employ themachine learning model that satisfies an end condition and of which theevaluation does not satisfy an acceptance condition.